🕷️️ Job Radar • SCRAPING

upwork.com 🟡 2026-05-08

🔹 High-Volume Image Download and Structured Storage
👤 Client: 🇺🇸 United States Member since 2023-08-24
💰 Price: $500
🚩 Problem: Organize bulk image downloads from UC Berkeley into a structured directory and produce a detailed manifest.
📦 Existing: [Stack]

Specifications:

[Target] - Download approximately 3.5 million TIFF or JPEG images.
[Method] - Implement authenticated download pipeline with rate-limiting and error-handling.
[UI/UX] - Not specified
[Stack] - Python, requests, CSV handling, AWS S3 (if applicable)
[Security] - Use provided credentials for institutional access agreement; handle sensitive data securely.
[Format] - Manifest CSV including source identifier, batch ID, frame number, and file size.

Workflow:

Set up authenticated download pipeline using provided UC Berkeley credentials.
Implement rate-limiting and error-handling according to licensor's terms of service.
Download images (TIFF or JPEG) and store in structured directory hierarchy by batch identifier.
Record metadata for each image: source identifier, batch ID, frame number, file size in CSV manifest.
Verify file integrity; re-download any failed files.
Provide weekly progress reports including download count, error rate, estimated completion date, issues encountered.
Deliver complete image collection on external drive or AWS S3 as specified.